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ABSTRACT 

Schoolwide projects may use Chapter 1 resources to 
improve the entire educational program of a school. In this respect 
they differ from other Chapter 1 projects in which resources are used 
to provide additional se, -vices only to students identified as 
educationally disadvantaged. It is possible that educationally 
disadvantaged students, who are the focus of P.L. 100*297, will 
receive less direct educational benefit in a schoolwide project. To 
protect against this, schoolwide projects are subject to additional 
accountability requirements: each schoolwide project is required to 
demonstrate that it is more effective than a traditional Chapter 1 
progrcUQ in icproving the achievement of its educationally 
disadvantaged students. If greater effectiveness cannot t% 
demonstrated after a 3-year period, a school will lose its status as 
a schoolwide project and revert to a traditional project. This paper 
discusses the options available to schools to satisfy ^he 
accountability requirement for schoolwide projects. Its purpose is to 
provide general practical guidance in selecting and implementing an 
approach that will fit local circumstances. (EVL) 
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Meeting the Accountability Requirement 
for Schoolwide Projects 



Alan Davis and Mary R. Quilling 



Introduction 

Schoolwide projects may use Chapter 1 resources to improve the entire educational 
program of a school In this respect they differ from other Chapter 1 projects in 
which resources are used to provide additional services only to students identified 
as educationally disadvantaged It is possible that educationally disadvantaged 
students, who are the focus of PX. 100-297, will receive less direct educational 
benefit in a schoolwide project To protect against this, schoolwide projects are 
subject to an additional accountability requirement each schoolwide project is 
required to demonstrate that it is more effective than a traditional Chapter 1 
program in improving the achievement of its educationally disadvantaged students. 
If greater effectiveness cannot be demonstrated aftwT a three year period, a school 
will lose its status as a schoolwide project and revert to a traditional project. 

This paper discusses the options available to schools to satisfy the accountability 
requirement for schoolwide projects. Its purpose is to provide general practical 
guidance in selecting and implementing an approach that will fit local 
circumstances. Local edui.ation agencies are encouraged to contact their Chapter 1 
Technical Assistance Center and state educanon agency for more concrete 
assistance in planning aaC conducting their accountability studies. 



Two AO jroac hes to Demonstrating Accountability 

The law (PL 100-297, Sec. 1015.e) and regulations (Federal Register, May 19, 1989, 
21765-6) offer two approaches to meeting the accountability requirement for 
schoolwide projects. At the end of the third project year, the average ^ 
achievement gains of educationally disadvantaged students in the school must be 
compared to the gains of: 

(a) a comparable group of Chapter 1 children in the districi as a whole; or 

(b) Chapter 1 children in the school during the three years prior to 
implementation of the schoolwide project. 

In this paper, we refer to the first approach as the Other Schools Compauson. and 
to the second as the Same School Comparison. 

1 Either the mean or the median can be used. The method must be specified in 
advance and used consistently throughout. 
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In cither comparison, the gains of educationally disadvantaged students in the 
schoolwide project must exceed the gains of the comparable students. Secondary 
schoolwide projects may demonstrate accountability with lower dropout rates, 
decreased retention rates, or increased graduation rates instead of greater 
achievement gains as long as achievement levels during the three schoolwide 
project years are not lower than the three years preceding the schoolwide project. 

We -ill discuss each approach in detail Before we do. we will address some 
preliminary questions common to both approaches: 

o What subject areas must be included in an accountability study? 

o What measures of achievement may be used? 

o On what comparisons must the schoolwide project exceed the 
comp^trison group? 

o What grades must be included? 

Each of these questions will be discussed in turn. 

Subject Area Focus 

In comparing the achievement gains of educationally disadvantaged students in 
schoolwide projects with gains of the comparison group, in what areas of 
achievement should the comparison be made? 

Simply stated, the purpose of the accountability requiremt it is to determine 
whether students are learning more in a schoolwide project than they would in a 
Chapter 1 project targeted at individual studesits. To determine what areas of 
achievement should be included, one must ask: What areas of instruction would be 
provided to these students through Chapter 1 in the absence of a schoolwide 
project? In keeping with Chapter 1 regulations, the targeting of subject areas must 
be supported by a needs assessment, and reflected in the application for the 
schoolwide project. 
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Example 1 

The student needs assessment in Central School District has indicated a need for 
Chapter 1 instruction in both reading and mathematics. Chapter 1 services in 
non-schoolwide project schools are being provided in both areas. When Taft 
Elementary became a schoolwide pro^ct, the staff wanted to set a schoolwide 
goal to improve reading, but did not set a goal for math. Both math and 
reading were identified as needs for the rest of the district for the first year of 
the schoolwide project Is the schoolwide pro^ct accountable for improved 
gains in both reading and math? 

Ye& In the absence of the schoolwide project educationally disadvantaged 
students at Taft would have received Chapter 1 supported instruction in reading 
and in math, so the school is accountable for gains in tM>th subjects. 



Selection of Outcome Measures 

What kinds of measures should be included? 

The LEA must designate one or more valid and reliable measures of student 
achievement in basic and a dvanced skills in its application that it will use to 
determine whether the accountability requirement has been met At a minimum, 
measures must include those tests used by the LEA to report aggregated 
performance for the Chapter 1 evaluation reporting system in the appropriate 
subject(s)i Additional measures may be used subject to the approval of the SEA. 
These may include criterion or domain referenced tests, applied performance 
measures, and other measures of achievement that are objective in the sense that 
they are administered and scored using procedures to minimize effects of rater 
exi^ctation or bias. 

Once an LEA has specffied an accountability measure in the plan for the 
schoolwide p.oject. it must collect data and report the results. Oii the other hand, 
it may amend its application at any time to si^cify additional measures to be used 
in the future. Comparisons based on all measures are subject to the rule that a 
comparison favoring the comparison group on one measure must be more than 
offset by a comparison favoring the schoolwide project on another, as discussed 
later in this paper. 

LEAs are encouraged to use multiple measures of achievement in meeting the 
accountability requirement. Relying on a single measure when important decisions 
rest on the outcome can encourage teaching narrowly to the format and content of 
the test The use of several m-asures allows students to demonstrate their 
achievement in a variety of ways and at different points in time. Tne use of 
applied performance measures, such as writing samples and math problem-solving 
demonstrations, is encouraged because such performances promote thoughtful 
instruction and require the integration and application of advanced skills. 
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According to the Chapter 1 regulations (May 1989, Section 20a3d£). comparisons 
must be hised upon achievement gains. When nonned measures are used, gains for 
grades two and above must be measured over a 12 month period. Gains on grade- 
specific measures that are not vertically equated (such as local writing samples) 
may be measured from fall to spring 



Example 2 

Urban School District provides Chapter 1 instructton in reading and language 
arts in grades l-S, Students in grade 1 are assessed using an informal reading 
inventory and a performance cbecklisi' of language skills. The district 
administers a nationally normed achievemestt test bAttcry to all students in 
grades 2-12 each spring. Gains on the Reading Comprehension and Total 
Language tests are aggregated and reported through the national Chf.pter 1 
evaluation reporting system. The district also administers a writing sample in 
the spring of grades 3 and 7 and a state minimum competency test in the spring 
of grade 6i In addition, students complete a self concept inventory in grade 5. 
The LEA would like to include all of these measu; as in meeting thv 
accountability requirement for the schoolwide Chapter 1 project at Central 
Elementary. 

Discussion: The nationally noimcd achievement test roust be included. The self 
concept inventory cannot be included, because it is not a measure Oi: 
achievement. The writing sample is an appropriate measure, since it corresfK}nds 
to the language arts goal of Chapter 1 in the district, but it can only be used if 
it is administered pre and post so that individual studsnt gains can be computed 
and aggregated. The state minimum competency test is administered at only one 
point in time, so it cannot be included. The informal reading inventory and 
performance checklist used at grade 1 are appropriate, but need to be expressed 
as gains. To do this, a pretest score must be obtained, although this need not 
involve format testing when it is apparent that students are cor pletely lacking 
the sicills to be measured 



Criteria for Successful Comparisons 

When goals include more than one area of achievement, or when more than one 
achievement measure is used, on what comparisons must the schoolwide project 
exceed the comparison group? 

If a schoolwide project establishes goals in more than one area, or if several types 
of measures are employed, several comparisons may be involved. For example, a 
project that sets out to improve achievement in reading and math in both basic and 
advanced skills could involve a minimum of four comparisons. 
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Since each comparison involves some amount of error, it is quite possible that a 
schooiwide project that is truly effective will not exceed the comparison group on 
every single compari»>n. It is sufficient that the "pre|K)nderance of the evidence" 
favor the schoolwide project To demonstrate this, any comparison in which the 
comparison group exceeds the schoolwide project must be more than offset by a 
comparison favoring the schoolwide pro^cL 

When comparisons involve the same standard metric (such as NCE scoresX the 
magnitude of the difference may be considered in determining whether a positive 
comparison offsets a negative comparison. For example, if the schcralwide project 
exceeds the comparison group by 4 NCEs in advanced skills in math but trails the 
comparison group by 1 NCE in basic skills in math, the positive comparison more 
than offsets the negative one. 

When different metrics are used, the count of positive comparisons must exceed the 
count of negative comparisons. For example, if the schoolwide project exceeds the 
control group in average improvement on a local writing sample, but trails the 
comparison ^roup in improvement on a state basic skills test and on a nationally 
normed achievement test, then the schoolwide project has failed to exceed the 
comparison group on thi majority of comparisons. 

The LEA must identify the comparisons it ^.11 make in advance in the application 
for the schoolwide project With the exception of secondary schools, comparisons 
must involve gains in basic skills and more advanced skills in at least one subject 
area. Comparisons may be made separately by grade level or aggregated across 
grade level 



Example 3 

Urban School District stated in its application that the accountability 
requirement for the schoolwide project at Central would be based upon annual 
gains on a nationally-normed t^t iu Total Language Arts and Reading 
Comprehension, and on fall-spring gains on a writing sample given in grade 3 • • 
only. Because the wtiting sample was given at only one grade, comparisons were 
made for each grade separately. The results are shown below: 
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Average Gains 
Of Educattooaliy Disadvantaged Students 



Tot Language 
(NCEs) 



Reading Comp. 
(NCEs) 



Writing 
(Av. Gain) 



Ceatnl Canfnrixoa 



Central Compvisoa 



Ceatral Codaparison 



2 
3 
4 
5 
6 



IS 

u 

21 

21 



U 

Z2 

u 

3.7 
Id 



J6 

M 

33 
21 
23 



15 
L9 



09 



The table presents 11 comparisons between the schoolwide project and the 
comparison group: 10 involving NCE gains on nationtUy normcd tests, and one 
involving average gains on a holtsticaliy scored writing sample. Of these 11 
comparisons, 7 (underlined) favored the schoolwide project Since the majority 
of comparisons favor the schoolwide project, the accountability requirement is 
satisfied, and the project is eligible for renewal 



Grades to Include 

What grade levels must be included in the comparison? 

A schoolwide project must be designed to upgrade the educational program in all 
the grades of the school (Regulatory Comment, Federal Register. May 19. 1989. 
p. 21792)l However, the comparisons required by the accountability provision 
apply only to the. : grades served by Chapter 1 in the local education agency as a 
whole or in the school prior to the initiation of the schoolwide project. If students 
in a particular grade in the schoolwide project would be served in Chapter 1 in the 
absence of the schoolwide project, then that grade must be included in the 
accountability comparison. In grades below the second grade, comparisons based 
upon average NCE g-lns must be aggregated separately from grades 2 - IZ 



7 



Ezampie 4 

Rural County School District providi^ Chapter 1 instruction to students in 
grades K-l Hurrison Elementary School, a K>5 school, has a schoolwide 
Chapter 1 project To meet the accountability requirement. Rural County will 
a>nduct grade by grade comparisons of student achievement gains for 
educationally disadvantaged students in grades K-4. In grades K and 1, Rural 
County proposes to give individually administered tests of receptive and 
eipressive language in the fall and the spring, and report gains in the total raw 
score. In grades 2-4, Rural County will use annual NCE gains on nationally 
normed tests. Grade S will not be included in the comparison, because fifth 
graders would not receive Chapter I services in the absence of the schoolwide 
project 



The Other Schools Comparison 

The Other Schools Comparison involves comparison of the gains of educationally 
disadvantaged students in the schoolwide project with gains of a comparable group 
of Chapter 1 children served in traditional projects. Since the purpose of the 
comparison is to determine whether a schoolwide project is more effective than 
traditional projects, the comparison group must be made up only of students in 
traditional Chapter 1 projects, and should not include students in other schoolwide 
projects. 

The comparison will be valid only to the extent that similar groups of students are 
compared. At a minimum, this means that the comparison of gains will be for 
students (a) identified by the same criteria and (b) in the same grades. The same 
tests and testing cycles must be used for both groups. If no such comparison group 
exists within the local educational agency, the Same School Comparison must be 
used instead. 

A significant advanT^ge of the Other Schools Comparison is that it allows the LEA 
to change tests or add additional measures without undue difficulty. LEAs can 
change tests or norms as often as they like, so long as the change affects the 
schoolwide project and comparison schools the same, and procedures are followed 
to allow the computation of gains during the year of the change. The Same School 
Comparison, on the other hand, presents a difficult problem when the LEA wants 
to change from whatever measures were used in the initial baseline period. 

The primary disadvantage of the Other Schools Comparison is that other schools 
may present an unfair comparison because of differences in the students they 
serve. Because schools hosting schoolwide projects must be schcmls serving a high 
concentration of students from low-income homes, these schools are generally more 
likely to face environmental obstacles to learning, such as lower attendance, less 
parent involvement and higher mobility. Schools serving low income areas may 
have a harder time attracting top faculty whose experience and prestige allow 
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them mobility within the system. These characteristics make it harder to raise 
achtevement 

The effect of statistical regression on achievement scores may also work to the 
detriment of the schoolw:de project. More affluent schools are generally higher 
achieving schools. The ist of a uniform district-wide selection criterion to 
identify Chapter 1 students will identify students in a higher achieving school who 
deviate more from the mean of the group from which selection takes place than 
the use of the same criterion in a low-achieving school. Consequently, regression is 
likely to contribute more to the gains of the comparison group if the same 
procedures and criteria arc used to identify educationally disadvantaged students 
in the schoolwide project 



Selecting Students in ^he Schoolwide Project for Comparison 

Differential regression, described above, can have a substantial effect on the 
outcome of the Other Schools Comparison. The problem is the same as that 
encountered by LEAs who used Model B of the older Title I Evaluation and 
Reporting System (TIERSX which involved comparing the gains of Title I students 
to the gains of comparable students in non-Title I schools. Although it would 
appear reasonable to identify comparable students by matching their scores or by 
applying the same cutoff score in both sets of schools, these approaches result in 
non-equivalent groups when the school populations themselves are not equivalent. 

The regression problem can be minimized by selecting groups so that they represent 
the same proportion of the larger pool of students from which they were selected . ^ 
To do this, find the number of students served in Chapter 1 in a given subject area 
in the LEA by grade (not including schoolwide projects). Divide that by the 
number of students in each grade in Chapter 1 schools where services in that 
subject are provided (not including schoolwide projects). This procedure yields tne 
proportion of students in Chapter 1 in each grade for a subject area. 

Next, find the number of students in each grade of the schoolwide project and 
multiply it by the corresponding proportion found above. This number represents 
the number of students to be included in the accountability study for this grade. 
Rank students by the same types of scores used to select students in other Chapter 
1 schools, omitting those who would not be eligible for Chapter 1 services on other 
grounds (e.g, those already receiving comparable servicesX up to the number 
required for that grade. Repeat this procedure for each grade and subject area. 
Note that this procedure will not yield the same students identified as 
educationally disadvantaged for purposes of funding or annual evaluation. It will, 
however, yield a sample for the accountability comparison that is less biased 
against the schoolwide project. 



1 This was precisely the approach used in TIERS Model R See Talmadge, 
Wood, C and Camel N. (1981). User's Guide: E SEA Title 1 Program Evaluation, p 
47. Washington. DC US Department of Education. 
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Example 5 

Adams Elemeotary School, a K-S school, has a schoolwide project There 
are 10 other Chapter 1 schools in the LEA. Chapter 1 reading instruction is 
provided in each of these 10 schools in grades K-3. In addition, four of the 
schools offer math instruction through Chapter 1 in grades 2-S. 

The LEA uses different criteria to select students for Chapter 1 in reading and 
math. In reading, students must score below the 3Sth percentile on a nationally 
normed test, and be rated *^ignif icantly low" in classroom performance by the 
classroom teacher. In math, students must score below the level of minimal 
acceptable performance on a math test developed by the LEA. 

To use the Other Schools Comparison, the LEA will first identify the proportion 
of students served in Chapter 1 reading and Chapter 1 math in the other 
schools, and then identify a conesponding proportion of students within Adams 
Elementary who w°"^*^ ^ served in Chapter 1 reading and math in the absence 
of the schoolwide project. The proportions for the LEA in reading are shown 
below: 



Chapter 1 Reading Enrollment as a Proportion 


of Total Enrollment of Chapter 1 Schools 






Chap 1 


Percent 


Grade Total N 


Reading 


Reading 


K 621 


128 


20^0 


1 638 


142 


222% 


2 617 


186 


302% 


3 5« 


144 


24X)% 



Now, the students in Adams must be ranked in order of need using the same 
selection criteria used in other schools. In reading, these include teacher 
judgments of student performance and scores on nationally normed tests. 
Students who would not be served in Chapter 1 for other reasons will bt 
omitted. The first 20^ of Kindergarten students listed, the first 722% of first 
graders, and so on, will constitute the comparison group for reading. The 
procedure will then be refuted in grades 2-5 for math. 



Thirri Year vs. All Three Years 

The comparison is to be made at the end of the three year period. At that time, 
the comparison may be for all gains during the three year period or between gains 
during the third year only. Taking into account the imperfect reliability of gains 
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in any given year (esi^cially ;n small projects), including gains for all three years 
generally provides a more stable indicator of impact On 'He other hand, if the 
project improves during the three year period, it should be able to rely upon the 
third year results. 

The Same School Comparison 

A schoolwide project may satisfy the accountability requirement by demonstrating 
after three years that the achievement gains of educationally disadvantaged 
students in the school exceed the average achievement gains of comparable 
educationally disadvantaged students in the same school in the three years prior to 
the start of the schoolwide project. 

This option is appealing when there are problems finding equivalent comparison 
groups within a district However, constructing equivalent historical comparison 
groups often poses significant difficulties that must be considered carefully. 
Ideally, to conduct the Same School Comparison, the school should remain 
relatively constant for six years in respect to: 

o the outcome measures administered, 

o the testing interval used, and 

o the demographic characteristics of the student population. 



Establishing the 3- Year Baseline 

Before the passage of P.L 100-297, most Chapter 1 projects measured achievement 
gains from fall to spring, and did not report gains for basic and advanced skills 
separately. Schools that established schoolwide projects soon after the passage of 
the new legislation may find it difficult or impossible to establish a baseline of 12- 
month gains in basic and advanced skills against which to compare subsequent 
gains. To do so in most cases would require re-examining the scores of each 
individual student, determining gains in basic and advanced skills on a 12-month 
testing cycle, and then computing new average gains for each grade for each of 
three years before the initiation of the schoolwide project Note that simply using 
the average scores of two spring testing (Mints will not accomplish the same result 
because the averages will not be restricted to those students with both pre end 
posttest scores. Note also that should the LEA compare annual gains of Ihe 
schoolwide project to fall-spring gains in previous years, the schoolwide project is 
very likely to fail in the comparison because gains measured fall-spring tend to be 
higher. 
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Example 6 

Adams Elementary School became a schoolwide project at the $ame time that the 
LEA moved from fall-spring to annual testing in Chapter L Adamis is very 
different from other Chapter 1 schools in the LEA, and elects to be compared to 
its previous performance rather than to other schools. But since future testing 
will be done only once a year, previous fall-spring gains are not a fair 
comparison. 

To recalculate annual gains, the LEA Chapter 1 coordinator worked with a 
clerk to locate the scores of Chapter 1 students for the three previous years. 
Calling the first of the three yea'.-$ Tear l." they found the spring scores of 
Chapter 1 students in both basic and advanced skills for that year. They then 
searched testing records for the s^' ring of the previous year to find matching 
pretest scores for thesie students. Matchitrg scores were located for about 70% of 
the students. Averaging the two sets of spring scores, they calculated the 
average spring-spring gain for Year 1, and repeated the process for Year 2 and 
Year 3^ Then, for each grade level they averaged together Year 1, Year 2, and 
Year 3 to find the average for the baseline fwrod Those averages became the 
standard for their Same School Comparison. 



Identifving Students .or Comparison 

A same-school comparison will be valid only to the extent that equivalent groups 
are comp: red across time. To accomplish this, the schoolwide project must ideraif y 
students who would have been served in Chapter 1 in that school in the absence of 
a schoolwide project. It must employ the same procedures tor selection used during 
the previous three years, selecting students in the same grades as in the past, using 
the same selection criteria. 

If changes occurred during the baseline period in the grades included in the 
project or the selection criteria employed, then bot^ comparison sets should ue re- 
constructed employing the most restrictive service parameters. 



Example 7 

Brook School, a K-6 school, served students scoring below the 30th percentile in 
grades 1-5 in a Chapter 1 reading program for two of the three years before 
becoming a schoolwide project The year before becoming a schoolwide project, 
it dropi^d grade 5, and continued serving 1-4. For the past three years, the 
school has operated as a schoolwide project with an objective to improve reading 
in all grades, and has considered all students below the SOth percentile as 
educationally disadvantaged Using the most restrictive parameters, the 
comparison for School A will include only students identified as below the 30th 
percentile in reading in grades 14 both before and after becoming a schoolwide 
project. 
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Third Year vs Three-Year Average 



Gains for the three years of the baseline period are averaged togt:her to establish 
the criterion for the Same School Comparison. After three years, the schoolwide 
project may choose to compare the baseline average to the gains of its students in 
thp third year only, or to the average of gains across the three years of the project 



Example 8 

Taf t School provided Chapter I services in reading in grades K-3 before 
becoming a schoolwide project Ta't chose to use the San^e School Comparison 
to satisfy the accountability requirement During the three years preceding the 
schoolwide project, gains in grades K and 1 were reported in increases in the 
number of skills mastered. In grades 2 and 3, annual NCE gains were reported 
in both basic and advanced skills. R^ults for the baseline period and for the 
first year of the project are shown below. To simplify the table, only gains in 
basic skills are displayed: 



Average Gains for Educationally Disadvantaged Students 



Baseline Year Project Year 

Grade 12 3 Average 1 



K 18L1 173 192 1&2 ia6 

1 121 152 13.7 U7 135 

2 1.6 -12 22 a9 14 

3 33 4.1 1.7 3.0 22 



Since the gains in different gradw involve different tyj^s of scores, gains for 
all four grades cannot be averaged together. Instead, comparisons are made 
grade by grade. From the table, it can be seen that gains in the first year of the 
schoolwide project exceed the baseline average in grades K and 2, but not in 
grades 1 and 3 Since the first year involved major changes in grouping and 
instruction, gains in subsequent years are expected to \x higher. The school may 
base its comparison entirely on the third year of the project 



Changes in Tms 

The Same School Comparison is most va'.id when gains are based upon the :»ame 
tests (or equivalent forms of the same tests) throughout the six year comparison 
period- On the other hand, LEAs will continue to select new measures, and 
publishers of norm-referenced tests will provide new aorms. Use of the Same 
School Comparison cannot lock in testing practices indefinitely. 
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The year a new test is adopted poses a special problem. The LEA may use 
equating tables from a study satisfying Chapter 1 standards for test equating to 
adjust pre-test scores for the year of adoption of the new test. If no such study 
has been conducted, then students in the schoolwide project must be tested with 
both the old and new instrument during the adoption year for the new test if the 
project wishes to retain the option of averaging gains over the three ptoject years. 
If only the gains from the third year are to be used for the accountability 
comparison, then a change in tests is simple, so long as It does not occur during the 
third project year. 

If the new test and the previous test are nationally normed, then gains for 
sub^squent years can be used without adjustment, based on the assumption that 
NCE gains are comparable from one test to another. 



Example 9 

After administering the Old Standard Achievement Test for 5 years, Central 
School District changed to the New Improved Test from a different publisher. 
The change occurred during the second year of a new schoolwide project No 
equating study was available fron either publisher. The year of the change, the 
LEA admiiyistered the new test to all students in the spring. It also 
administered the old test to those students in the schoolwide project who were 
identified as educationally disadvantaged for purposes of the Same School 
Comparison study and who had a pretest from the previous spring. Gains for 
Year 2 were based on the old test; the new test given that spring became pretest 
scores for Year 1 By following this procedure, the LEA left itself the option of 
using the average of three years of gains for the schoolwide project, rather than 
relying exclusively on the gains for Year 3 using the new test. 



Selecting the Accountability Approach 

For many schcolwide projects, circumstances will dictate the choice of 
accountability approach. Schoolwide projects in sparsely populated areas or in 
educational agencies in which all schools are schoolwide projects may have to use 
the Same School approach because no comparable traditional Chapter 1 projects are 
available as a comparison group. Other schoolwide projects without a th-ee-year 
history of Chapter 1 must compare themselves to oiher schools. For most, the 
choice involves weighing considerations of validity and effort. 

Because it uses cur, jnt data, the Other Schools Comparison is preferable when 
circumstances in the LEA have changed appreciably over the six year period This 
approach is unaffected by change ; in tests, testing intervals, and data editing 
procedures when gains for the third year are compared The Same School 
Comparison, on the other hand is compromised by changes in the population 
served by the school due to desegregation policies or shifts in boundaries of school 
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attendance areas. Differences in student selection procedures, such as inclusion of 
LEP students or service to retained students may also weaken the Same School 
Comparison. 

Both approaches may involve time-consuming re-calculation of gains for the 
school-wide project to insure equivalence of comparison groujs. However, these 
efforts are generally greater for the Same School Comparison, particularly when 
the school has changed its testing cycle, tests, selection criteria, or targeted grades 
during the six-year period. Retrieval of data 3 to 6 years old is likely to be a 
frustrating and time consuming task, particularly if *he data are not already 
computerized The cost of searching records by hand and the likely compromises 
in data quality a;e further cautions against a quick choice of the Same School 
approach. 

The Same School Comparison has the advantage of helping to motivate staff by 
calling on everyone to raise achievement more than it has been raised in the past, 
and it avoids invidious comparisons with other current projects. The Same School 
Comparison may yield a more valid result when traditional schools in the district 
are too different from tht school hosting the schoolwide project to provide an 
equivalent comparison groun. ^ner or later, however, schools using this approach 
will face changes in tests oj norms that may affect gains in ways that cannot be 
predicted The approach also poses a constraint on those LEA' who would prefer 
to introduce very different types of assessments than those used in the past 



When Neither Aporoach is Feasible 

There are local situations in which neither the Same School Comparison nor the 
Other Schools Comparison can be used For example, consider a small rural 
education agency in which the only Chapter :* school at each level has already 
operated as a schoolwide project under previous legislative provisions. It may well 
be that there is no baseline of data previous to the establishment of the schoolwide 
project for a Same School Comparison, yet there are no other Chapter 1 schools in 
the agency for the Other Schools Comparison. In this situation (which must be 
confirmed by the SEA), the accountability requirement is accomplished through the 
same evaluation procedure used to determine whether the school must develop a 
plan for program improvement 

The local education agency must demonstrate that the schoolwide project has 
shown substantial progress toward meeting the desired outcomes described in its 
application, and has shown improvement in the aggregate performance of 
educationally disadvantaged children in the school over a three year period. This 
determination is made yearly as par» of the annual review, but the determination 
for the accountability requirement is made at the end of the three year period. At 
that time, the determination may be based on gains for the full three year period, 
or on gains during the third year only, using the "preponderance of evidence" 
approach described previously The comparison must include both basic and 
advanced skills. If the evidence does not demonstrate both an improvement in 
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aggregate performance and substantial progress toward meeting the desired 
outcomes described in the application, then the project witt not be authorized to 
continue schoolwide. 
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